Efficient Search in Big Data using Refinder

Inthis paper, we present a context-based information refinding system called ReFinder. It leverages human’s natural recall characteristics and allows users to refind files and Web pages according to the previous access context. ReFinderrefinds information based on a query-by-context model over a context memory snapshot, linking to the accessed information contents. Context instances in the memory snapshot are organized in a clustered and associated manner, and dynamically evolve in life cycles to mimic brain memory’s decay and reinforcement phenomena. We evaluate the scalability of ReFinder on a large synthetic data set. The experimental results show that consistent degradation of context instances in the context memory and the ones in user’s refinding requests can lead to the best refinding precision and recall. An 8-week user study is also conducted to examine the applicability of ReFinder. Initial findings show that time, place, and activity could serve as useful recall clues. On average, 15.53 seconds are needed to complete a refinding request with ReFinder and 84.42 seconds with other existing methods. Big Data concern large-volume, complex, growing data sets with multiple, autonomous sources. With the fast developmentof networking, data storage, and the data collection capacity, Big Data are now rapidly expanding in all science and engineeringdomains, including physical, biological and biomedical sciences. This paper presents a HACE theorem that characterizes the featuresof the Big Data revolution, and proposes a Big Data processing model, from the data mining perspective. This data-driven modelinvolves demand-driven aggregation of information sources, mining and analysis, user interest modeling, and security and privacyconsiderations. We analyze the challenging issues in the data-driven model and also in the Big Data revolution. Some further possible improvement of ReFinder is also discussed at the end of the paper.